Intelligent Regression Analytics

AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.687362	2.13
	2_DecisionTree	Decision Tree	logloss	2.38929	12.22
	3_Linear	Linear	logloss	0.744102	11.82
	4_Default_Xgboost	Xgboost	logloss	0.688909	7.32
	5_Default_NeuralNetwork	Neural Network	logloss	0.740673	3.69
	6_Default_RandomForest	Random Forest	logloss	0.683196	5.28
the best	Ensemble	Ensemble	logloss	0.678106	1.49

AutoML Performance

AutoML Performance Boxplot

Features Importance (Original Scale)

Scaled Features Importance (MinMax per Model)

Spearman Correlation of Models

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

0.4 seconds

Metric details

	score	threshold
logloss	0.687362	nan
auc	0.5	nan
f1	0.717949	0.48
accuracy	0.56	0.48
precision	0.56	0.48
recall	1	0.48
mcc	0	0.48

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.687362	nan
auc	0.5	nan
f1	0.717949	0.48
accuracy	0.56	0.48
precision	0.56	0.48
recall	1	0.48
mcc	0	0.48

Confusion matrix (at threshold=0.48)

	Predicted as 0	Predicted as 1
Labeled as 0	0	11
Labeled as 1	0	14

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

11.0 seconds

Metric details

	score	threshold
logloss	2.38929	nan
auc	0.551948	nan
f1	0.777778	0
accuracy	0.68	0
precision	0.636364	0
recall	1	0
mcc	0.416598	0

Metric details with threshold from accuracy metric

	score	threshold
logloss	2.38929	nan
auc	0.551948	nan
f1	0.777778	0
accuracy	0.68	0
precision	0.636364	0
recall	1	0
mcc	0.416598	0

Confusion matrix (at threshold=0.0)

	Predicted as 0	Predicted as 1
Labeled as 0	3	8
Labeled as 1	0	14

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 3_Linear

<< Go back

Logistic Regression (Linear)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

10.3 seconds

Metric details

	score	threshold
logloss	0.744102	nan
auc	0.233766	nan
f1	0.717949	0.401834
accuracy	0.56	0.401834
precision	0.56	0.401834
recall	1	0.401834
mcc	0	0.401834

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.744102	nan
auc	0.233766	nan
f1	0.717949	0.401834
accuracy	0.56	0.401834
precision	0.56	0.401834
recall	1	0.401834
mcc	0	0.401834

Confusion matrix (at threshold=0.401834)

	Predicted as 0	Predicted as 1
Labeled as 0	0	11
Labeled as 1	0	14

Learning curves

Coefficients

feature	Learner_1
feature_1	0.22
intercept	0.219517
feature_2	-0.171419

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 4_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

5.2 seconds

Metric details

	score	threshold
logloss	0.688909	nan
auc	0.448052	nan
f1	0.717949	0.461612
accuracy	0.56	0.461612
precision	1	0.544404
recall	1	0.461612
mcc	0.327327	0.544404

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.688909	nan
auc	0.448052	nan
f1	0.717949	0.461612
accuracy	0.56	0.461612
precision	0.56	0.461612
recall	1	0.461612
mcc	0	0.461612

Confusion matrix (at threshold=0.461612)

	Predicted as 0	Predicted as 1
Labeled as 0	0	11
Labeled as 1	0	14

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

Top-10 Best decisions for class 0 (Fold 1)

Top-10 Worst decisions for class 1 (Fold 1)

Top-10 Best decisions for class 1 (Fold 1)

<< Go back

Summary of 5_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

2.1 seconds

Metric details

	score	threshold
logloss	0.740673	nan
auc	0.220779	nan
f1	0.717949	0.39283
accuracy	0.56	0.39283
precision	0.56	0.39283
recall	1	0.39283
mcc	0	0.39283

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.740673	nan
auc	0.220779	nan
f1	0.717949	0.39283
accuracy	0.56	0.39283
precision	0.56	0.39283
recall	1	0.39283
mcc	0	0.39283

Confusion matrix (at threshold=0.39283)

	Predicted as 0	Predicted as 1
Labeled as 0	0	11
Labeled as 1	0	14

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 6_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

3.9 seconds

Metric details

	score	threshold
logloss	0.683196	nan
auc	0.564935	nan
f1	0.722222	0.42674
accuracy	0.6	0.42674
precision	1	0.657309
recall	1	0.343799
mcc	0.386859	0.657309

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.683196	nan
auc	0.564935	nan
f1	0.722222	0.42674
accuracy	0.6	0.42674
precision	0.590909	0.42674
recall	0.928571	0.42674
mcc	0.168623	0.42674

Confusion matrix (at threshold=0.42674)

	Predicted as 0	Predicted as 1
Labeled as 0	2	9
Labeled as 1	1	13

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
1_Baseline	3
2_DecisionTree	1
6_Default_RandomForest	2

Metric details

	score	threshold
logloss	0.678106	nan
auc	0.590909	nan
f1	0.777778	0.461848
accuracy	0.68	0.461848
precision	1	0.648989
recall	1	0.366272
mcc	0.416598	0.461848

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.678106	nan
auc	0.590909	nan
f1	0.777778	0.461848
accuracy	0.68	0.461848
precision	0.636364	0.461848
recall	1	0.461848
mcc	0.416598	0.461848

Confusion matrix (at threshold=0.461848)

	Predicted as 0	Predicted as 1
Labeled as 0	3	8
Labeled as 1	0	14

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back